Model: allenai/OLMo-7B-0424-hf

Layer: 31. Neuron Index: 9634

gate+_in+

gate+_in-

gate-_in+

gate-_in-

Frequency: 14.67%. Frequency: 67.70%. Frequency: 0.29%. Frequency: 17.34%.
hook_post:
Max: 3.48;
Min: 0.00;
Avg: 0.16.
hook_post:
Max: 0.00;
Min: -5.12;
Avg: -0.18.
hook_post:
Max: 0.00;
Min: -0.49;
Avg: -0.08.
hook_post:
Max: 1.58;
Min: 0.00;
Avg: 0.20.
hook_pre_linear:
Max: 3.11;
Min: 0.00;
Avg: 0.25.
hook_pre_linear:
Max: 0.00;
Min: -3.27;
Avg: -0.48.
hook_pre_linear:
Max: 2.64;
Min: 0.00;
Avg: 0.83.
hook_pre_linear:
Max: 0.00;
Min: -7.24;
Avg: -1.20.
hook_pre:
Max: 4.10;
Min: 0.00;
Avg: 0.95.
hook_pre:
Max: 4.13;
Min: 0.00;
Avg: 0.66.
hook_pre:
Max: 0.00;
Min: -2.32;
Avg: -0.24.
hook_pre:
Max: 0.00;
Min: -5.43;
Avg: -0.50.
swish:
Max: 4.04;
Min: 0.00;
Avg: 0.71.
swish:
Max: 4.06;
Min: 0.00;
Avg: 0.46.
swish:
Max: 0.00;
Min: -0.28;
Avg: -0.09.
swish:
Max: 0.00;
Min: -0.28;
Avg: -0.15.

Prototypical activations for case gate+_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15




Prototypical activations for case gate+_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15




Prototypical activations for case gate-_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15



Prototypical activations for case gate-_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15